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ABSTRACT 

Recognition  languages  are  developed  for  the  better 
communication  of  the  challenged  people.  The 
recognition  signs  include  the  combination  of  various 
with  hand  gestures,  movement,  arms  and  facial 
expressions  to  convey  the  words  thought.  The 
languages  used  in  sign  are  rich  and  complex  as  equal 
as  to  languages  that  are  spoken.  As  the  technological 
world  is  growing  rapidly,  the  sign  languages  for 
human  are  made  to  recognised  by  systems  in  order  to 
improve  the  accuracy  and  the  multiply  the  various 
sign  languages  with  newer  forms.  In  order  to  improve 
the  accuracy  in  detecting  the  input  sign,  a  model  has 
been  proposed.  The  proposed  model  consists  of  three 
phases  a  training  phase,  a  testing  phase  and  a  storage 
output  phase.  A  gesture  is  extracted  from  the  given 
input  picture.  The  extracted  image  is  processed  to 
remove  the  background  noise  data  with  the  help  of 
threshold  pixel  image  value.  After  the  removal  of 
noise  from  the  image  and  the  fdtered  image  to  trained 
model  is  tested  with  a  user  input  and  then  the 
detection  accuracy  is  measured.  A  total  of  50  sign 
gestures  were  loaded  into  the  training  model.  The 
trained  model  accuracy  is  measured  and  then  the 
output  is  extracted  in  the  form  of  the  mentioned 
language  symbol.  The  detection  mechanism  of  the 
proposed  model  is  compared  with  the  other  detection 
methods  such  as  Hidden  Markov  Model(HMM), 
Convolutional  Neural  Networks(CNN)  and  Support 
Vector  Machine(SVM).  The  classification  is  done  by 
means  of  a  Support  Vector  Machine(SVM)  which 
classifies  at  a  higher  accuracy.  The  accuracy  obtained 


was  99  percent  in  comparison  with  the  other  detection 
methods. 
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I.  INTRODUCTION 

The  natural  way  of  exchanging  information  among 
the  deaf  and  dumb  is  sign  language.  In  order  to 
overcome  the  difficulties  faced  by  deaf  and  dumb 
people  when  they  communicate  with  normal  people 
sign  recognition  language  was  invented.  There  are 
three  types  of  sign  languages  Indian  Sign  Language, 
British  Sign  Language  and  American  Sign  Language. 
Only  5%  of  Indian  deaf  and  dumb  people  get 
education  in  order  to  increase  educating  the  deaf  and 
dumb  people  sign  languages  are  used.  Indian  Sign 
Language  has  its  own  specific  syntax,  grammar, 
alphabets,  words  and  numerals.  The  gestures  are  made 
with  the  help  of  combination  of  fingers,  hand,  wrist 
movements  and  elbow  movements.  There  are  two 
types  of  aspects  one  is  by  positioning  the  finger  and 
another  one  is  by  changing  the  orientations.  The  sign 
language  needed  to  be  well  structured  for  a  higher 
level  of  understanding.  The  sign  language  is  process 
involves  image  acquisition  with  image  classification, 
facial  expressions  are  ignored  as  they  are  ineffective 
of  the  substantial  expressions  in  it.  for  improving  the 
accuracy  and  a  new  method  has  been  proposed.  The 
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method  includes  the  combination  of  the  various  data  and  classifies  it.  This  method  not  implemented  the 


expressions  involving  sign  nature.  The  method 
detected  with  a  99%  accuracy. 

II.  RELATED  WORKS 

An  Sandeep  kaul  and  manider  singh[2015]  designed  a 
model  with  SigML  and  HamNoSys.  The  JA  SIGML 
URL  App  was  developed  inorder  to  verify  the  HamNo 
input.  The  inputs  were  in  the  form  of  HamNoSys 
notations.  The  accuracy  of  the  animated  signs  is  tested 
with  the  help  of  “Indian  Sign  Language  Dictionary”. 
The  main  disadvantage  of  the  model  no  concurrent 
words  are  recognized.  A  novel  technique  focusing  on 
transformation  content  of  sign  language  images  by 
Nalini  Yadav  et.  al  [2015].  The  goal  of  the  proposed 
classification  technique  provided  a  faster  retrieval 
with  higher  accuracy.  Kumad  Tripathi  et.  al  proposed 
a  system  by  combining  Orientation  Histogram  and 
Principal  Component  Analysis(PCA)  which  reduced 
the  dimension  features  after  extraction.  Combined  the 
correlation  and  Euclidean  distance  which  gave  an 
higher  accuracy.  [2015],  Asha  Thalange  and  Dr.  S.  K. 
Dixit.  [20 16]  Proposed  two  new  feature  extraction 
techniques  naming  them  as  Combined  Orientation 
Histogram  and  Statistical(COHST)  features  and 
Wavelet  Features  for  recognition  of  static  signs  of 
numbers  .  It  was  implemented  using  Neural  networks. 
The  proposed  system  was  with  an  accuracy  of  98.12% 
of  recognition.  Surbhi  rahat  and  ujwal  gandat  [2017] 
proposed  a  vision  based  method  inducing  processing  , 
feature  extraction,  segmentation  and  classification. 
The  vision  based  method  enhances  with  various 
approaches  that  converts  the  text  into  voice  module 


Training  Phase 


inducing  of  sentence  conversation. 

Neela  harish  and  Poonguzhali  [2015]  proposed  a 
system  which  included  flex  sensors  and  accelerometer 
fixed  to  the  fingers  and  wrist.  These  sensors  are  fitted 
with  the  help  of  glove.  The  text  were  converted  with 
the  help  of  the  electric  signals  produced  interms  of 
voltage  and  the  voice  output  were  produced  and  sent 
to  the  voice  module  producing  appropriate  words. 
Prof.  Prasanth  G  et.  al  [2015]  designed  a  system  for 
processing  video  to  speech  will  include  frame 
formation  from  videos  and  region  of  interst  with 
language  knowledge  base  using  correlational  based 
approach  than  the  audio  conversion.  The  mapping  of 
text  and  relevant  animated  gestures  from  the  database 
are  implemented.  It  was  not  designed  for  American 
Sign  Language.  A  smart  glove  technique  using  Intel 
Galileo  Gen2  IOT  kit  for  real  time  gesture  recognition 
by  Abinandhan  Das  et.  al  .  The  device  included  both 
sign  language  recognition  and  gesture  recognition 
system[2016]. 

III.  PROPOSED  SYSTEM 

The  proposed  system  developed  to  efficiently  detect 
the  gestures  recognized  at  a  high  accuracy.  The 
Gesture  Recognition  System  (GRS)  consists  of  three 
phases  through  which  the  output  for  the  input  gesture 
is  observed.  The  three  phases  involve  mainly  a 
training  phase,  a  dataset  training  storage  phase  and  a 
testing  phase. 


Testing  Phase 


classification 


Fig.  1  Architecture  of  GRS 


A.  Training  phase 

The  entire  document  should  be  in  Times  New  Roman 
or  Times  font.  Type  3  fonts  must  not  be  used.  Other 
font  types  may  be  used  if  needed  for  special  purposes. 


The  training  phase  is  primarily  provided  with  an  input 
symbol  recognition  dataset.  The  dataset  involves  the 
various  symbols  that  have  different  pictures  that 
denote  the  various  gestures  for  English  alphabets  and 
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numbers.  The  dataset  also  involves  various 
background  noises  i.e.  data  is  not  preprocessed. 

i.  Gesture  extraction 

The  input  symbols  involve  various  background  data 
which  may  be  considered  as  noise  or  outliers.  Then 
with  the  help  of  the  color  intensity  variation  and  edge 
detection,  gestures  are  recognized.  The  Gestures  are 
recognized  with  the  edge  detection.  Further,  the  image 
needs  preprocessing  for  the  accurate  gesture 
detection. 


Fig.2  Extraction  of  Gesture 
ii.  Image  Processing 

The  image  with  a  detected  edge  is  then  moved  for  edge 
filtering.  An  image  filter  for  background  subtraction  is 
implemented.  The  background  image  is  omitted  with  the 
removal  of  the  components  outside  the  edge  detected.  The 
edge  contains  some  of  the  input  data  that  may  not  be 
recognized  for  filtering.  A  threshold  value  is  respect  to  the 
intensity  of  the  data  that  is  processed.  The  portion  of  the 
image  that  matches  the  intensity  value  is  retained  and 
others  are  filtered.  Finally,  an  image  observed  is  converted 
into  the  input  format  which  is  supportable  to  be  given  as 
the  input  dataset  for  the  testing  phase. 


Fig.3  Processing  of  image 


B.  Storage  Phase 

The  input  images  extracted  with  a  gesture  symbols  is 
stored  onto  the  database  storage.  A  dynamic  data 
storage  database  is  used  here  where  the  new  gestures 
can  be  feed  as  the  input.  The  gestures  stored  onto  the 
database  are  moved  onto  the  testing  phase  to  test  the 
accuracy  of  the  model. 


U  f*  -ij 
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Fig.4  Different  signs  of  ISL 
C.  Testing  Phase 

The  testing  phase  implemented  with  the  data  from  the 
user.  The  user  data  is  given  by  a  web  camera.  The 
web  camera  provides  the  gesture  of  the  input  hand 
sign.  The  edges  of  the  fingers  are  detected  in  the 
system.  The  points  of  the  edge  are  recognized.  The 
background  images  rather  than  the  edges  detected  are 
removed  by  the  model.  The  gesture  recognized  is  the 
classified  with  given  input  datasets  and  returns  the 
maximum  probable  gesture.  It  returns  the  various 
gestures  that  result  above  the  threshold  value.  The 
comparison  module  compares  the  classified  gestures 
with  the  input  pattern.  Enhance  the  recognized  pattern 
with  nearest  sign  image.  The  image  that  matches  the 
nearest  value  is  then  returned  to  the  database.  The 
database  data  that  matches  the  nearest  value  for  the 
sign  is  returned  as  the  output  with  a  high  accuracy. 


Fig.5  Gesture  testing  of  ISL 
D.  Experimental  evaluation 

The  experimental  setup  is  done  by  means  of  Java  IDE 
with  python  environment.  The  four  detection  methods 
are  built  in  python  code.  The  simulation  environment 
is  build  for  1,  50,  100  and  500  datasets.  Two  efficient 
metrics  were  observed  in  the  environment.  Firstly,  the 
time  taken  to  detect  the  sign  that  has  been  recognized. 
The  GRS  model  tended  to  detect  the  sign  feed  within 
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the  shortest  amount  of  time  as  given  table  1.1. 
Secondly,  the  detection  accuracy  is  measured  with  the 
help  of  the  precision  and  recall  metrics  which  were 
observed  that  all  the  signs  given  into  the  system  were 
observed  accurately. 

TABLE  I 

GESTURE  DETECTION  TIME  FOR  DIFFERENT 
METHODS 


recognition  techniques  such  as  HMM,  CNN  and 
SVM,  the  GRS  technique  concluded  to  have  an  higher 
classification  and  recognition  of  various  symbols  in 
ISL.  The  model  classified  500  symbols.  The  GRS  will 
be  a  boon  to  the  challenged  people  and  to  gain  higher 
standards  in  education.  The  future  work  is  to  develop 
a  mobile  application  for  the  sign  recognition  system. 
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E.  Conclusions 

The  Gestures  Recognition  System  is  developed 
through  this  model.  The  GRS  model  recognizes  the 
input  signal  with  a  complete  accuracy.  The 
Combination  of  the  phases  induced  the  system  with 
larger  symbol  detection  mechanisms.  Comparison  of 
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